Large scale applications are often designed in a modular fashion. Commonly, these modules will have well-defined or even standardized input/output formats and objectives. These well-defined input/output format and objectives allow collaborative effort amongst multiple, independent researchers and engineers to contribute to the development community by solving or attempting to address modularized problems and propose various optimization techniques or heuristics (e.g., as “software modules”) to effectively and efficiently solve the problems. As a result, well-established domains may be saturated with several software approaches or hardware tools from various vendors to solve any single problem.
For example, a typical computer-aided design (CAD) application for very large scale integrated circuits (VLSI) may have several “steps” implemented as cooperative but logically separate software modules: 1) a hardware description language (HDL) translation module that generates a netlist with primitive logic gates and their connection network, 2) a place module that maps logic gates to corresponding suitable locations on a semiconductor and 3) a routing module that places wires that connects the logic gates. Because the input and output format of typical CAD applications are well defined, the particular CAD domain has become reasonably saturated, and several integrated applications are commercially available which ostensibly perform the same tasks, at varying efficiencies. Selecting an appropriate, high-performing application or even a compatible configuration of modules from multiple applications to perform a particular CAD design can become a difficult proposition. Likewise, selecting well-performing applications or configurations to perform other tasks can be similarly complicated.
Specifically, it may not always be immediately discernible which tool or approach will be the solution with a desirable level of performance for solving a particular problem or performing a specific task. Often, several solutions will each have fervent support among its developers and users. Conventionally, a high-performing solution for a particular task or problem may be accurately achieved by applying a wide selection of tools to solve a particular problem or to perform a particular task, comparing the results, and selecting the tool which achieved the best results. Unfortunately, purchasing operating licenses for every available tool can be extremely cost-intensive.
Although the cost of some tools may be partially mitigated by the availability of evaluation licenses, in many cases the evaluation versions will have severe restrictions on usage and/or limited features. Furthermore, effectively testing and comparing solution candidates typically requires a large scale infrastructure that can execute multiple tools in parallel. Moreover, because the algorithms these tools implement are usually computing and memory intensive, the execution of multiple tools may be prohibitively time intensive. Furthermore, amassing the specific hardware and software resources for sufficiently large infrastructures of information technology is notoriously expensive, and configuring the resources to perform cooperatively can be enormously complex. Finally, even when the interface is well defined, and the infrastructure is in place, integrating multiple third party tools into a single, seamless workflow is non trivial and typically requires operators with significant tool-specific knowledge, and, even then can be extremely time consuming and error-prone to use.